Augmented reality systems (also referred to as “mixed reality systems”) provide users with a view of a physical, real world environment having elements that are augmented by computer-generated or extracted real world input, such as video, sound, or graphics. Computer-generated enhancements are layered atop an existing reality in order to make it more meaningful through the ability to interact with it. Augmented reality has been developed into “apps” for use on mobile computing devices, such as smartphones or tablet computers, to blend digital components into a displayed view of the real world in such a way that they enhance one another. Augmented reality can also be achieved with more immersive hardware, through the use of head-mounted displays (HMDs) that blend the virtual imagery with the real-world view, or through a video that merges virtual objects with a real-time camera captured view of the real environment.
A virtual reality system is another type of system used for enhancing a user's interface with a computing device. Virtual reality is an artificial, computer-generated simulation or recreation of a real life environment or situation. This technology may use HMDs, or employ large monitors and projector-based environments (called world-fixed displays), to generate realistic images, sounds, and/or other sensations that simulate a user's physical presence in a virtual reality environment. Virtual reality systems can replace a real-world environment with a simulated one, whereas an augmented reality system can enhance one's perception of reality.
An important component of augmented reality and virtual reality systems is the user interface. An example user interface is a “wand controller,” which can function like a cross between a motion controller and a pen. Such a user interface or wand controller may be elongated in shape for use by a user to “point” to a virtual object in the augmented or virtual reality environment. The system may recognize that the virtual object is being pointed to. Subsequently, the user may depress a button on the wand controller to thereby allow the user to use the “wand” to move or otherwise engage with the object.
Selection and manipulation of virtual objects in world-fixed displays, such as cave automatic virtual environment (CAVE) systems, are limited by the narrow avenue of interaction possibilities provided by wand controllers. Issues, such as occlusion and the lack of realistic haptic feedback caused by such a user interface, hinder the ability to perform realistic interactions. Accordingly, there is a need for improved systems and techniques for allowing users to interact with virtual objects within augmented and virtual reality environments.